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Abstract. Mathematical problem solving typically involves manipulating visual symbols (e.g., 
equations), and prior research suggests that those symbols serve as diagrammatic 
representations (e.g., Landy & Goldstone, 2010). The present work examines the ways that 
instructional design of student engagement with these diagrammatic representations may 
impact student learning. We report on two studies. The first describes systematic cross- 
cultural differences in the ways that teachers use mathematical representations as diagrammatic 
supports during middle school mathematics lessons, finding that teachers in two higher 
achieving regions, Hong Kong, and Japan, more frequently provided multiple layers of support 
for engaging with these diagrams (e.g. making them visible for a longer period, using linking 
gestures, and drawing on familiarity in those representations), than teachers in the U.S., a lower 
achieving region. In Study 2, we experimentally manipulated the amount of diagrammatic 
support for visually presented problems in a video-based fifth-grade lesson on proportional 
reasoning to determine whether these multiple layers of support impact learning. Results 
suggest that learning was optimized when supports were used in combination. Taken together, 
these studies suggest that providing visual, temporal, and familiarity cues as supports for 
learning from a diagrammatic representation is likely to improve mathematics learning, but that 
administering these supports non-systematically is likely to be overall less effective. 
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1 Introduction 


Teaching students mathematics that is flexible, transferrable, and connected across 
topics is crucial to high quality instruction; however, despite decades of agreement to 
this pedagogical goal, many students struggle with such mathematical thinking (e.g., 


Polya, 1954; Bransford, Brown & Cocking, 2001; National Mathematics Panel, 
2008). Given the difficulties students experience in learning mathematics, a key goal 
is to improve instruction. 

Improving mathematics instruction requires an understanding of the cognitive 
processes involved in acquiring mathematics knowledge and skills. In this paper, we 
highlight one challenge to thinking mathematically: learning to perceive mathematical 
problems and symbolic equations as sets of relationships (arithmetic, proportional, 
inequality, etc.) (see Richland, Stigler & Holyoak, 2010). The ability to perceive the 
relational structure of mathematics allows the problem solver to more easily draw 
connections across problems or mathematical ideas, and to think more conceptually 
about mathematics. 

Mathematical representations are diagrammatic if they convey structural 
properties through their spatial and perceptual attributes. As an illustration, in the 
equation 3 + (4—5), a learner must know to subtract 5 from the 4 before adding three 
because the parentheses convey priority in carrying out the operations. Thus, part of 
the students’ task in working with mathematical representations is to perceive the 
relevant features and structural relations that are embedded in their perceptual 
instantiations. Effective mathematical instruction, therefore, requires attention to how 
perceptual representations are presented in order to support students’ understanding of 
mathematical concepts (Kellmann & Massey, 2013; Richland & McDonough, 2010). 

Considering mathematical equations and symbolic representations as diagrams 
allows one to formulate insights into how to best support learners in identifying the 
core relationships within these representations. Diagrams can use spatial cues and 
sparse representations to highlight relationships rather than simply depict iconic 
information (Ainsworth, 2006; Michal et al. 2016), however it is not the case that 
students learn from any and all experiences with a diagram (Rau, 2017). Rather, it is 
clear that not only must the diagram be informative and relevant, but also pedagogical 
practices for supporting students’ thinking and must improve the likelihood that 
students notice and attend to the key relationships being depicted (e.g., Richland & 
McDonough, 2010). To improve students’ attention to relationships, we can draw on 
strategies for ensuring that students learn from diagrams to inform mathematical 
pedagogy. 

In the present paper, we begin by reviewing principles deriving from perceptual 
learning, mathematics, and reasoning literatures to highlight strategies for how to 
improve attention to relationships within diagrams: a) use visual representations, 
make them visible while discussing them and subsequent representations, b) use hand 
movements (linking gestures) to move between instructional diagrams, and c) draw 
on material that is familiar to learners (see below for more detail). We then describe 
two sets of data suggesting that combining these strategies systematically may be the 
most potent way to improve student learning from mathematical diagrams. First, we 
describe an analysis of cross-national data collected as part of the Third International 
Mathematics and Science Study (TIMSS, Hiebert et al, 2003) showing that teachers in 
two regions that outperform the United States, Hong Kong and Japan, used these 
pedagogical principles in combination more systematically than did U.S. teachers. 
These data suggest both that these strategies must be considered in combination, 


rather than as separable practices, but also that these may be correlated with student 
learning. In a second study, we report an experimental design in which we tested the 
efficacy of them being used together. The results support the correlational data 
identified in study one, together providing consistent indications that these are 
important pedagogical practices that can support students’ attention during 
engagement with mathematical diagrams, and that doing so has consequences for 
student learning even when the mathematical diagram and the audio-stream of the 
lessons are identical. 


1.1. Perceptual Learning in Mathematics 


Though mathematics has traditionally been viewed as involving conceptual learning, 
there is evidence that mathematics learning is highly perceptual (e.g., Kellman & 
Massey, 2013). For instance, mathematical concepts are frequently represented in the 
form of symbolic notations, which themselves contain perceptual attributes that are 
connected to structural properties. As an illustration, Landy and Goldstone (2010) had 
subjects solve simple equations that were presented either spatially consistent with the 
order of operations (e.g., solving 3. + (4 —5)) or spatially inconsistent (e.g., 
solving 3 + (4 - 5)) — these authors found that the spatial distance between 
influenced problem solving: Subjects were less accurate at solving simple equations 
when they were spaced in ways that were inconsistent with the order of operations, 
suggesting that even adults represent simple equations as types of diagrams. 

As another illustration of perceptual learning in mathematics, interventions that 
support perceptual learning processes have shown some promise for improving 
students learning outcomes. For example, Kellman and colleagues have developed 
visual matching exercises that engage learners in linking different representations of 
mathematical concepts. In these exercises, students do not formally solve problems or 
conduct calculations; rather, students learn to identify the attributes that connect 
different mathematical representations. Despite never formally solving problems, 
students who engage in these linking activities are more accurate and at later problem 
solving than students that do not engage in them (Kellman et al. 2008; Kellman, 
Massey, & Son, 2010). A related perceptual learning intervention that allows students 
to perform physical manipulations of equations that are consistent with the 
grammatical rules of algebra has shown promise for supporting students’ algebraic 
understanding (Ottmar, Landy, & Goldstone, 2012). 

Because mathematics involves perceiving the relevant structure in 
representations, diagrammatic supports that highlight structure can be a powerful tool 
to promote mathematical understanding and fluency (Rau, Aleven, & Rummel, 2009; 
Rittle-Johnson, Star, & Durkin, 2009). At the same time, simply providing diagrams 
may not result in successful learning (Rau, 2017). Often domain leaners fail to notice 
relevant correspondences between representations unless highly supported in doing so 
(e.g., Alfieri, Nokes-Malach & Schunn, 2011; Gick & Holyoak, 1980, 1983; Richland 
& McDonough, 2010). Children and domain novices (both characteristics of k-12 
school children) are most susceptible to missing key elements of comparisons and 
attending to irrelevant salient features that impede relational thinking, in part due to 


low cognitive processing resources (e.g., Richland, Morrison & Holyoak, 2006). We 
next review research on how to support reasoning with diagrams in mathematics. 


1.2. Diagrammatic Supports 


While diagrams can serve as effective cognitive supports for learning in mathematics, 
students need support in comprehending diagrammatic representations before they 
can benefit from them. Understanding of visualizations in mathematics can be 
supported by cognitive aids that highlight relevant structural properties in the 
representations. The science of learning has made advances in understanding of how 
students best learn with diagrammatic representations. These principles for supporting 
diagrammatic fluency are discussed, below. 

Making Representations Visible Simultaneously. Research suggests that learning in 
general is facilitated by the use of simultaneous diagrammatic representations 
(Gadgil, Chi, & Nokes, 2013; Gentner, 2010; Matlen et al. 2011; Richland & 
McDonough, 2010). In the domain of mathematics, Rittle-Johnson and Star (2007) 
found that middle-school age students were more likely to improve in solving 
algebraic equations when students compared multiple worked out equations as 
compared to studying them in isolation. Simultaneous presentation prompted students 
to compare the two domains and highlighted the relevant structural attributes of the 
equations. Other mathematics studies have shown learning gains when two visual 
representations are displayed simultaneously versus sequentially, leading to gains in 
procedural knowledge, flexibility, and conceptual understanding (e.g., Richland & 
Begolli, 2016; Rittle-Johnson, Star & Durkin, 2009). 

Use spatial organization to highlight key relations. Whenever two representations 
are compared there are many similarities and differences that could be attended to. 
Learning is enhanced when the spatial organization of the representations highlights 
the alignments. For example, Kurtz & Gentner (2013) found participants were faster 
and more accurate at detecting differences in skeletal structures when two skeletal 
images were presented in the same orientation relative to when they were presented in 
a symmetrical orientation. Further, Matlen, Gentner, and Franconeri, (2014, in prep) 
found that placing images in direct spatial alignment, such that a student need not 
move through one object to find alignments with another, optimized the speed and 
accuracy with which analogies were processed. In contrast, impeded alignments were 
slower and led to more errors. 

Use linking gestures to move between spatial representations. Linking gestures are 
hand movements that move between two (or more) representations that are being 
compared, sometimes highlighting the specific alignments between these 
representations, and other times simply providing support for noticing the relevance 
of one representation to another (Alibali & Nathan, 2007, Alibali, Nathan & Fujimori, 
2011; Richland, 2016). For instance, Richland and McDonough (2009) provided 
undergraduates with examples of permutation and combination problems that 
incorporated visual cueing, such as gesturing back and forth between problems and 
allowing the examples to remain in full view, versus comparisons that did not 
incorporate visual cueing. Students who studied the problems with visual cueing were 


more likely to succeed on difficult transfer problems. Linking gesture use is 
correlated with high mathematics learning in students (Richland, in 2016) and teacher 
gesture is well known to improve learning outcomes (see Goldin-Meadow, 2003). 
Don’t overload learners’ cognitive resources. Reasoning with multiple 
representations requires adequate working memory (WM) and executive function 
(EF) resources, leading to reasoning failure and lower rates of learning when 
resources are overloaded or non-functioning (e.g., Richland, Morrison & Holyoak, 
2006; Walz et al, 2000; Cho et al, 2007). When the contributions of working memory 
(WM) and inhibitory control (IC) were examined separately on children’s successful 
learning and transfer from a classroom lesson based on an instructional analogy, we 
found that both explained distinct variance for predicting improvements in procedural 
knowledge, procedural flexibility, and conceptual knowledge after a 1-week delay 
(Begolli et al, in press) WM & IC were less predictive at immediate post-test, 
suggesting that these functions are not simply correlated with mathematics skill, but 
may be particularly important in the process of durable schema-formation (Begolli & 
Richland, 2015). To reduce cognitive load during comparison, visual representations 
from familiar examples or domains can be used when possible in order to help 
students understand unfamiliar examples or domains (Duit, 1991). 


1.3. Implementing Supports in Practice 


The above supports provide guidance for instructional decisions in classrooms. 
However, many mechanisms of learning operate simultaneously in everyday 
classrooms, and may augment or undermine each other, meaning that theories that 
explain learning in isolation may actually differ from those that explain learning in 
classrooms. Despite a large research base on what supports reasoning with 
diagrammatic representations, little research has explored the combinatorial use of 
supports, and this is particularly true in the context of authentic classrooms learning 
environments. 

One reason why this may be important is that some supports may seemingly 
contradict one another. For instance, supports 1-3 described above are hypothesized to 
function because they reduce the cognitive processing load on reasoners to notice and 
draw inferences based on similarities between the representations; however, it is 
possible that adding simultaneous visual representations, spatial alignments, and 
gestures to process simultaneously could instead augment processing load. Thus, 
studying the integration of these principles is key to understanding how supports 
function in combination, resulting in more informed recommendations for how to best 
structure diagrammatic supports for classroom learning. 


1.4 The Present Studies 


The present studies examine the combinatorial use of diagrammatic both descriptively 
and experimentally. Study 1 consisted of a cross-cultural examination of the use of 
diagrammatic supports in middle school classrooms. The study builds on prior 
research by Richland, Zur, and Holyoak (2006) who coded the frequency of 
diagrammatic supports using a sample of eighth-grade mathematics lessons taught in 


the U.S., Hong Kong, and Japan from the Third International Mathematics and 
Science Video Study (TIMSS, Hiebert et al, 2005). These authors found that U.S. 
teachers regularly use comparison and contrasting cases in mathematics instruction, 
yet they do so without using diagrammatic supports as frequently as East Asian 
teachers. However, in this research, the frequency of co-occurrence of the supports 
was not examined. Thus, the present investigation re-analyzed lessons from the U.S., 
Hong Kong, and Japan to understand the combinatorial use of diagrammatic 
principles in these regions. In Study 2, we experimentally examine the impact of 
different combinations of diagrammatic supports on middle school students’ 
mathematics learning. 


Z Study 1 


2.1 Method 


Videodata were collected as part of the Third International Mathematics and Science 
Video Study (TIMSS, Hiebert et al, 2005) through a randomized probability sample 
of all eighth-grade mathematics lessons taught in the U.S. and seven higher achieving 
regions internationally. These data were analyzed and reported in a previous study 
(see Richland, Zur, & Holyoak, 2007). The current study involved a re-analysis of 
codes from a set of thirty lessons that were randomly selected from the U.S., Hong 
Kong, and Japan. Each lesson was taught by a different teacher, and all verbalized or 
visually presented comparisons were identified and then coded for their presence of 
principles for supporting student comparison efforts. 


source VisAvail = the source domain of the comparison was visually available 
gestureComp = use of linking gestures for comparison 

visualAlignment = problems were spatially aligned 

sourceUnfamiliar = whether the source of the comparison was unfamiliar 


The data for the present study were re-analyzed to explore the extent to which 
diagrammatic supports were used in combination cross-culturally in western and 
eastern regions. Prior reports of this data indicated that teachers in Japan and Hong 
Kong were more likely to use diagrammatic supports than teachers in the U.S. 
(Richland, Zur, & Holyoak, 2007). Thus, in the present study we combined Japanese 
and Hong Kong lessons in the present analysis. 

The data set consisted of 588 previously coded analogies in 30 lessons from each 
of the three regions listed above (n = 10 each). Codes for the diagrammatic supports 
for each analogy relative to the total number of analogies presented were averaged 
within each lesson. 


2.2 Results 


To determine the extent to which diagrammatic supports co-occurred with one 
another, Pearson correlations were conducted between the supports within each 
region. The results of this analysis for East Asian and U.S. lessons are presented in 


Figures 1 and 2, respectively. As can be seen from Figure 1, in East Asian lessons, 
supports evidenced moderate to high positive correlations (ranging from .45 to .68), 
indicating that supports are used moderately often in combination. In addition, all 
correlations within East Asian lessons were statistically significant from a zero 
correlation (ps < .05). In contrast, correlations between supports in the U.S. were 
inconsistent in their direction (ranging from -.48 to .29; see Figure 2). Moreover, no 
correlations in U.S. lessons were statistically different from a zero correlation (all ps > 
.15). Though we view these findings as primarily descriptive, the results suggest that 
in addition to using diagrammatic supports less frequently in the U.S. than in East 
Asian countries (Richland, Zur, & Holyoak, 2007), U.S. teachers are also less likely 
to use supports in combination. 
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Fig. 1. Pearson correlations between diagrammatic supports in East Asian mathematics lessons 
(i.e., Hong Kong and Japan, N = 20). All correlations are statistically significant at alpha < .05. 
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Fig. 2. Pearson correlations between diagrammatic supports in U.S. mathematics lessons (N = 
10). No correlations are statistically significant (all ps > .15). 


sourceUnfamiliar 


Prior research suggested that U.S. teachers regularly use comparison and 
contrasting cases in mathematics instruction, yet they do so without adequately 
supporting students in drawing these connections (Hiebert, 2003; Richland, Zur & 
Holyoak, 2007). Importantly, teachers’ rates of supporting students in drawing 
connections between mathematical ideas or problems during problem solving was the 
single factor that differentiated all higher and lower achieving peer countries in the 
Trends in International Mathematics and Science 1999 Video Study (Hiebert, 2003). 
Though NCTM and disciplinary panels have long recommended helping students 
make mathematical connections (see National Mathematics Panel, 2008; Polya, 
1954), this is still a serious challenge for teachers (Hiebert, Stigler, Jacobs et al, 
2005). The present study suggests that U.S. teachers also use these supports less 
frequently in combination than in east Asian countries — this was particularly true 
when the source problem was unfamiliar. This finding contrasts to East Asian 
teachers, who were more likely to use supports when the source problem was 
unfamiliar. 


3 Study 2 


Study | revealed that teachers in two higher achieving regions, Hong Kong and Japan, 
more frequently provided multiple layers of support for engaging with these diagrams 
in systematic ways, such that if one support strategy were used another was often used 
(i.e. making representations visible for a longer period, spatial alignment between 

diagrams, using linking gestures, and drawing on familiarity in those representations). 


This correlated use of support strategies mapped onto student achievement patterns, 
such that teachers in Hong Kong and Japan used these practices more than teachers in 
the U.S., which is a lower achieving region. This trend was suggestive of a 
relationship to achievement, but not conclusive. Thus in Study 2, we experimentally 
manipulated the amount of diagrammatic support provided for visually presented 
problems in a video-based fifth-grade lesson on proportional reasoning, to determine 
whether these multiple layers of support impact learning. 


3.1 Method 


The present experiment, we independently manipulated the familiarity of a source 
example problem with the amount of diagrammatic support provided to assess the 
influence of familiarity and visual supports together, separately, and in comparison to 
instruction without either of them. Specifically, the design was a 2 (familiarity 
condition: Unfamiliar or Familiar) x 2 (support condition: All Support or No Support) 
between subjects randomized trial. 


Participants. Two hundred sixty-seven 5th grade students participated in this study. 
Forty-nine participants (18%) were excluded because they did not complete either the 
familiarity manipulation or one of the three math assessments. Of the remaining 218 
participants, 61 were in the Familiar-All Support condition, 50 were in the Familiar- 
No Support condition, 56 were in the Unfamiliar-All Support condition, and 51 were 
in the Unfamiliar-No Support condition (See Table | for demographic representations 
of students). The study was run in nine total classrooms in five schools in the Chicago 
area. Four of these schools were public charter schools, while one was a Catholic 
school. 


Table 1. Demographics of participating students in the analytic sample. 


Demographic Percent in Sample Number in Sample 
Total N = 218 
Females 58% 126 
African-American 16% 34 
Hispanic 57% 124 
White 13% 29 
Other Race(s) 14% 31 


Procedure. Classrooms were visited after permission from the school’s 
administration and teachers was granted. Each participating classroom was visited 
three times over a two-week period. Students were told that the goal of the study was 
to understand the best ways to teach kids math. 

Visit 1. The first visit to the classroom lasted approximately 1 hour. Students 
completed two baseline measures, and then were randomly assigned to either a 
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familiarity training condition or a no familiarity training condition. The baseline 
measures were the following:: 


1. Patterns of Adaptive Learning Survey (PALS). This 24-item measured goal 
orientation, and consisted of three scales (Mastery Goal Orientation, Performance- 
Approach Goal Orientation, and Performance-Avoid Goal Orientation) (Midgley et 
al., 2000). One question about students’ level of math anxiety (“Math makes me 
feel nervous”) and two questions about students’ long division abilities (“I’ve been 
taught long division before” and “I can do long division’) were added to the end of 
this survey. 

2. Content Knowledge Assessment. This assessment was a researcher-designed test 
consisting of 7 items that assessed students’ baseline level of knowledge of rate 
and ratio concepts and long division abilities (see Begolli & Richland, 2016 for test 
properties). 


On the content knowledge pre-test, students were instructed to attempt each 
problem and were asked to show all of their work, even if they weren’t able to get a 
final answer. The pre-measures took approximately 45 minutes for students to 
complete. 

After students completed the pre-measures, they were randomly assigned to one 

of the two familiarity conditions. Half of the students in each classroom were given 
long division instruction (Familiar condition) while the other half was given practice 
with long division problems (Unfamiliar condition). All students had been previously 
instructed in long division, these were simply opportunities to retrieve and strengthen 
the familiarity of these procedures. Students in both familiarity conditions were given 
a worksheet containing the same three long division problems. For students in the 
Familiarity condition, the first problem was worked out for them step-by-step, with 
instructions for each step. Students were asked to solve the second problem 
themselves, but were given those same step-by-step instructions with space next to 
each instruction for students to complete that step. Students were then asked to solve 
the third problem on their own. Students in the Control condition were given the 3 
long division problems and were simply asked to solve each problem and show their 
work. 
Visit 2. Researchers returned to each classroom 2-7 days later (mean= 4.3 days, 
median= 4.5 days), for a 90 minute session. Students were assigned to one of two 
video-based instruction conditions: All Supports or No Supports. Assignment was 
random but with the constraint that one problem on the baseline test was scored and 
used in order to minimize any differences between baseline performance across 
conditions. 

All of the videos contained the identical audio stream of information, and the 
lessons were the same teacher and classroom, but one video included more visual 
access to the mathematical diagrams on the board and to linking gestures, while the 
other video did not have these pedagogical supports. 

The lesson content involved a lesson about ratio, centered on a comparison 
between multiple ways that different students solved a word problem involving a set 
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of ratios. . Both video lessons began with a teacher asking students to solve a ratio 
problem any way they would like (see Table 2). Two students in the video were then 
asked to share the method they used to solve the problem. The first student in the 
video indicated that he used the Least Common Multiple (LCM) method to solve the 
problem. The teacher then solved the problem on the board using the LCM method 
described by the student. A second student in the video told the class how he used 
division to solve the problem. The teacher solved the problem on the board using the 
division method described by the student. Finally, the teacher discussed the 
definitions of rate and ratio, summarized the lesson, and compared the two solution 
methods. Students in the study completed problems and answered teacher questions 
in a packet along with students who appeared in the video lesson. 

The All Support and No Support videos differed in three ways (see Figure 3): 1) 
In the All Support video, the two methods of solving the problem (LCM and division) 
remained visible on the board throughout the lesson. In the No Support video, the 
LCM solution was not visible again once the discussion of the division method began. 
2) In the All Support video, these two solution methods were presented on the board 
in a parallel structure so that comparisons between the two solution methods could be 
made more easily. In the No Support video, the two solution methods were not 
shown at the same time, so this way of organizing the board could not support 
students in making comparisons between the two solution methods. 3) In the All 
Support video, the teacher used linking gestures while comparing the two solution 
methods. Gestures were not used in the No Support video. After the video lesson, 
students in the study completed an immediate post-test to assess how much they 
learned from the video lesson. Finally, students completed a 10-item survey that 
tested their level of engagement with the lesson. 


Table 2. Students were provided the following prompt (accompanied by the table, below) in 
each video lesson: Ken and Yoko shot several free throws in their basketball game. The result 
of their shooting is shown in the table. Who is better at shooting free throws? 


Shooter Shots Made Shots Tried 
Ken 12 20 
Yoko 16 25 
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Fig. 3. Screen shots of identical points in the AS and NS videos during which the teacher 
compared two solution strategies. Parallel organization on the board, two visible solution 
strategies, and gesture were used in the All Support condition, but not in the No Support 
condition. 


Visit 3. Researchers returned for a third visit 4-9 days later (mean = 6.7, median = 7). 
Students completed 3 tasks during this visit, which lasted 1 hour. As a group, 
students completed the d2 Test of Attention, a paper and pencil test that measures 
concentration and selective attention (Brickencamp & Zilmer, 1998). Next, students 
completed the delayed post-test to assess how much they remembered from the video 
lesson during our second visit. Finally, students completed a demographics 
questionnaire. 


3.2 Results 


Outcomes of interest in the present investigation concerned students’ performance on 
the content knowledge assessment. Specifically, we were interested in increases in the 
correct strategy use at post-test vs. pre-test (students’ use of either the LCM or 
division strategies), and decreases in the incorrect subtraction strategy from pre- to 
post-test. For this reason, we concentrated our analyses on the problems that required 
students to choose a strategy and solve the problem on their own (this analysis does 
not include student responses to multiple choice questions). Outcome scores represent 
an average across all non-multiple choice questions. 


Correct strategy use. To explore the presence of correct strategy use across 
conditions, we conducted separate 2 (familiarity condition) x 2 (support condition) 
between subjects ANOVAs on students’ gains from pre-test to post-test (Visit | vs. 
Visit 2) and from gains from pre-test to delayed post-test (Visit 1 vs. Visit 3), using 
the correct use of either the LCM or division strategy as the outcome variable. The 
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analysis for the pre- to post-test ANOVA revealed significant main effects of support 
condition (F(1,214) = 11.52, p = .001) and familiarity condition (F(1,214) = 4.36, p = 
.04), and a marginally significant interaction between support and familiarity 
conditions (F(1,214) = 3.11, p = .08) on correct strategy use. Games-Howell post-hoc 
tests revealed that these effects were primarily driven by the performance of students 
in the Familiar-All Supports condition, who performed significantly better than 
students in the three other conditions (ps < .02). 
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Fig. 4. Average percent change in correct strategy use from pre-test to delayed post-test by 
condition. 


At delayed post, the ANOVA analysis revealed a significant main effect of 
support condition (F(1,214) = 6.73, p = .01), no main effect of familiarity condition 
(F(1,214) = .90, p = .35), and a marginally significant interaction between support and 
familiarity conditions (F(1,214) = 2.76, p = .10) on correct strategy use. We show 
delayed post performance in Figure 4, as this time-point represents learning that is 
sustained over time, and is arguably the strongest test of our hypotheses. To explore 
the interaction, we conducted Games-Howell post-hoc tests to make comparisons 
between conditions. This analysis revealed that the interaction was driven primarily 
by higher performance in the Familiar-All Support condition relative to the Familiar- 
No Support condition (p = .02) and the Unfamiliar—No Support condition (p = .09). 
Decreases in use of the incorrect, subtraction strategy. A common incorrect 
strategy for comparing ratios involves the use of subtraction, where students subtract 
part of the whole (e.g., shots made from the total amount tried) (see Begolli & 
Richland, 2016). To explore the use of this strategy, we conducted a 2 (familiarity 
condition) x 2 (support condition) between subjects ANOVA on students’ decreases 
in use of the subtraction strategy from pre to post-test and from pre- to delayed-post- 
test. The ANOVA on decreases from pre to post-test revealed a main effect of support 
(F(,214) = 6.35, p= .01) but no effect of familiarity condition and no interaction 
between support and familiarity. Post-hoc Games-Howell tests revealed a marginally 
significant effect for students in the Familiarity-All Support condition to decrease 
their use of the misconception more often than students in the Unfamiliarity-No 
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Support condition (p= .06) 

The ANOVA on decreases from pre to delayed post-test revealed only a 
significant main effect of the support condition (F(1,214) = 7.98, p = .005) (see Figure 
5). Marginal differences in the decreased use of the subtraction strategy were found 
for pair-wise comparisons between the Familiar-All Support condition vs Unfamiliar- 
No Support condition (p = .10) and the Unfamiliar-All Support condition vs the 
Unfamiliar-No Support condition (p=.06). 
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Fig. 5. Average percent change for the incorrect subtraction strategy from pre-test to delayed 
post-test by condition. 


4 General Discussion 


The studied presented here involved an observational and experimental exploration of 
the use of diagrammatic supports in combination in middle school mathematics 
classrooms. Study 1 involved a cross-cultural examination of teachers’ use of 
diagrammatic supports and find that teachers in Hong Kong and Japan more 
frequently combine diagrammatic supports in mathematics lessons, whereas U.S. 
teachers combine supports less systematically in mathematics lessons. Study 2 
manipulated the amount of diagrammatic support for visually presented problems in a 
fifth-grade lesson on proportional reasoning. Results suggest that learning about 
proportions is optimized when supports are used in combination. Though we did not 
explore whether combinations of supports aid learners more than single supports, 
these studies together suggest that providing visual, temporal, and familiarity cues as 
supports for learning from a diagrammatic representation is likely to improve 
mathematics learning. 

The present research is an early attempt to explore how cognitive supports for 
diagrams interact in authentic environments, and is consistent with recent calls to 
explore instructional complexity in authentic contexts. For example, Koedinger, 
Booth, and Klahr (2013) estimate that there are on the order of trillions of 
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instructional decisions that must be made during the course of classroom teaching, 
and suggest that more investigations are needed that explore how instructional 
principles interact with one another, as well as how they interact with the content to 
be learned. These authors suggest that educational technology environments can used 
to test a large number of permutations of instructional combinations to address this 
problem (e.g., Koedinger, Booth & Klahr, 2013). Similarly, we use video 
methodology to deeply situate this work in authentic student learning environments 
that are complex and that routinely combine multiple pedagogical principles, while 
maintaining internal validity of our experimental approach. In doing so, we attempt to 
understand how instructional combinations impact learning of mathematics in both an 
internally and ecologically valid way. 

Though our approach directly examines the relationship between principle 
enactment and student learning, future work can more directly examine the issues that 
teachers confront during the course of enacting principles. For example, teachers must 
enact principles while they are attempting to hold both the content of the lesson and 
students’ understanding of the content of the lesson in mind — presumably a high 
demand on cognitive resources — nevertheless, little work has addressed how 
enactment of principles influences teacher cognition. Explorations of this issue in 
future work might better inform theory on how principles can be optimally used in 
applied contexts. Moreover, our future reports will examine relationships between 
student characteristics known to correlate with mathematics learning, such as anxiety 
and executive function, to exposure to combinatorial support use and learning. This 
work will shed light on ways in which diagrammatic supports interact with other 
factors in applied contexts. 
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